在本文中,我们介绍了一条神经渲染管道,用于将一个人在源视频中的面部表情,头部姿势和身体运动转移到目标视频中的另一个人。我们将方法应用于手语视频的具有挑战性的案例:给定手语用户的源视频,我们可以忠实地传输执行的手册(例如握手,棕榈方向,运动,位置)和非手术(例如,眼睛凝视,凝视,面部表情,头部移动)以照片真实的方式标志着目标视频。为了有效捕获上述提示,这些线索对于手语交流至关重要,我们以最近引入的最健壮和最可靠的深度学习方法的有效组合来建立。使用3D感知表示,将身体部位的估计运动组合并重新定位到目标签名者。然后将它们作为我们的视频渲染网络的条件输入,从而生成时间一致和照片现实的视频。我们进行了详细的定性和定量评估和比较,这些评估和比较证明了我们的方法的有效性及其对现有方法的优势。我们的方法产生了前所未有的现实主义的有希望的结果,可用于手语匿名。此外,它很容易适用于重新制定其他类型的全身活动(舞蹈,表演,锻炼等)以及手语生产系统的合成模块。
translated by 谷歌翻译
由于深度学习的出现,图像数据的最新技术对单眼3D面对重建的重建取得了令人印象深刻的进步。但是,它主要集中于来自单个RGB图像的输入,忽略以下重要因素:a)如今,感兴趣的绝大多数面部图像数据不是来自单个图像,而是来自包含丰富动态信息的视频。 。 b)此外,这些视频通常以某种形式的口头交流捕捉个人(公众对话,电视会议,视听人类计算机的互动,访谈,电影中的独白/对话等)。当在此类视频中应用现有的3D面部重建方法时,重建口腔区域的形状和运动中的伪影通常很严重,因为它们与语音音频不太匹配。为了克服上述局限性,我们提出了3D口表达的视觉语音感知重建的第一种方法。我们通过提出“口语”损失来做到这一点,该损失指导拟合过程,从而使3D重建的说话头的感知与原始录像相似。我们证明,有趣的是,与传统的具有里程碑意义的损失,甚至直接3D监督相比,口头损失更适合3D重建嘴运动。此外,设计的方法不依赖于任何文本转录或相应的音频,因此非常适合在未标记的数据集中培训。我们通过对三个大规模数据集的详尽客观评估以及通过两种基于网络的用户研究进行主观评估来验证方法的效率。
translated by 谷歌翻译
在本文中,我们介绍了一种新颖的深入学习方法,用于“野外”视频中演员的情绪状态的光学逼真操纵。所提出的方法基于输入场景中的演员的参数3D面表示,其提供来自头部姿势和面部表达的面部身份的可靠性解剖。然后,它使用新的深度域翻译框架,以符合他们的动态,以一致而合理的方式改变面部表情。最后,改变改变的面部表情用于基于特别设计的神经面渲染器光实际地操纵输入场景中的面部区域。据我们所知,我们的方法是第一个能够通过唯一用作操纵情绪的语义标记来控制演员的面部表情,同时保持与语音相关的唇部运动。我们进行广泛的定性和定量评估和比较,展示了我们的方法的有效性以及我们获得的特别有希望的结果。我们的方法为神经渲染技术的有用应用开辟了一种新的可能性,从电影后生产和视频游戏到照片逼真的情感化身。
translated by 谷歌翻译
Automatic fake news detection is a challenging problem in misinformation spreading, and it has tremendous real-world political and social impacts. Past studies have proposed machine learning-based methods for detecting such fake news, focusing on different properties of the published news articles, such as linguistic characteristics of the actual content, which however have limitations due to the apparent language barriers. Departing from such efforts, we propose FNDaaS, the first automatic, content-agnostic fake news detection method, that considers new and unstudied features such as network and structural characteristics per news website. This method can be enforced as-a-Service, either at the ISP-side for easier scalability and maintenance, or user-side for better end-user privacy. We demonstrate the efficacy of our method using data crawled from existing lists of 637 fake and 1183 real news websites, and by building and testing a proof of concept system that materializes our proposal. Our analysis of data collected from these websites shows that the vast majority of fake news domains are very young and appear to have lower time periods of an IP associated with their domain than real news ones. By conducting various experiments with machine learning classifiers, we demonstrate that FNDaaS can achieve an AUC score of up to 0.967 on past sites, and up to 77-92% accuracy on newly-flagged ones.
translated by 谷歌翻译
Accurate determination of a small molecule candidate (ligand) binding pose in its target protein pocket is important for computer-aided drug discovery. Typical rigid-body docking methods ignore the pocket flexibility of protein, while the more accurate pose generation using molecular dynamics is hindered by slow protein dynamics. We develop a tiered tensor transform (3T) algorithm to rapidly generate diverse protein-ligand complex conformations for both pose and affinity estimation in drug screening, requiring neither machine learning training nor lengthy dynamics computation, while maintaining both coarse-grain-like coordinated protein dynamics and atomistic-level details of the complex pocket. The 3T conformation structures we generate are closer to experimental co-crystal structures than those generated by docking software, and more importantly achieve significantly higher accuracy in active ligand classification than traditional ensemble docking using hundreds of experimental protein conformations. 3T structure transformation is decoupled from the system physics, making future usage in other computational scientific domains possible.
translated by 谷歌翻译
Variational autoencoders model high-dimensional data by positing low-dimensional latent variables that are mapped through a flexible distribution parametrized by a neural network. Unfortunately, variational autoencoders often suffer from posterior collapse: the posterior of the latent variables is equal to its prior, rendering the variational autoencoder useless as a means to produce meaningful representations. Existing approaches to posterior collapse often attribute it to the use of neural networks or optimization issues due to variational approximation. In this paper, we consider posterior collapse as a problem of latent variable non-identifiability. We prove that the posterior collapses if and only if the latent variables are non-identifiable in the generative model. This fact implies that posterior collapse is not a phenomenon specific to the use of flexible distributions or approximate inference. Rather, it can occur in classical probabilistic models even with exact inference, which we also demonstrate. Based on these results, we propose a class of latent-identifiable variational autoencoders, deep generative models which enforce identifiability without sacrificing flexibility. This model class resolves the problem of latent variable non-identifiability by leveraging bijective Brenier maps and parameterizing them with input convex neural networks, without special variational inference objectives or optimization tricks. Across synthetic and real datasets, latent-identifiable variational autoencoders outperform existing methods in mitigating posterior collapse and providing meaningful representations of the data.
translated by 谷歌翻译
Differentiable Architecture Search (DARTS) has attracted considerable attention as a gradient-based Neural Architecture Search (NAS) method. Since the introduction of DARTS, there has been little work done on adapting the action space based on state-of-art architecture design principles for CNNs. In this work, we aim to address this gap by incrementally augmenting the DARTS search space with micro-design changes inspired by ConvNeXt and studying the trade-off between accuracy, evaluation layer count, and computational cost. To this end, we introduce the Pseudo-Inverted Bottleneck conv block intending to reduce the computational footprint of the inverted bottleneck block proposed in ConvNeXt. Our proposed architecture is much less sensitive to evaluation layer count and outperforms a DARTS network with similar size significantly, at layer counts as small as 2. Furthermore, with less layers, not only does it achieve higher accuracy with lower GMACs and parameter count, GradCAM comparisons show that our network is able to better detect distinctive features of target objects compared to DARTS.
translated by 谷歌翻译
Deep learning techniques with neural networks have been used effectively in computational fluid dynamics (CFD) to obtain solutions to nonlinear differential equations. This paper presents a physics-informed neural network (PINN) approach to solve the Blasius function. This method eliminates the process of changing the non-linear differential equation to an initial value problem. Also, it tackles the convergence issue arising in the conventional series solution. It is seen that this method produces results that are at par with the numerical and conventional methods. The solution is extended to the negative axis to show that PINNs capture the singularity of the function at $\eta=-5.69$
translated by 谷歌翻译
The Government of Kerala had increased the frequency of supply of free food kits owing to the pandemic, however, these items were static and not indicative of the personal preferences of the consumers. This paper conducts a comparative analysis of various clustering techniques on a scaled-down version of a real-world dataset obtained through a conjoint analysis-based survey. Clustering carried out by centroid-based methods such as k means is analyzed and the results are plotted along with SVD, and finally, a conclusion is reached as to which among the two is better. Once the clusters have been formulated, commodities are also decided upon for each cluster. Also, clustering is further enhanced by reassignment, based on a specific cluster loss threshold. Thus, the most efficacious clustering technique for designing a food kit tailored to the needs of individuals is finally obtained.
translated by 谷歌翻译
Learning efficient and interpretable policies has been a challenging task in reinforcement learning (RL), particularly in the visual RL setting with complex scenes. While neural networks have achieved competitive performance, the resulting policies are often over-parameterized black boxes that are difficult to interpret and deploy efficiently. More recent symbolic RL frameworks have shown that high-level domain-specific programming logic can be designed to handle both policy learning and symbolic planning. However, these approaches rely on coded primitives with little feature learning, and when applied to high-dimensional visual scenes, they can suffer from scalability issues and perform poorly when images have complex object interactions. To address these challenges, we propose \textit{Differentiable Symbolic Expression Search} (DiffSES), a novel symbolic learning approach that discovers discrete symbolic policies using partially differentiable optimization. By using object-level abstractions instead of raw pixel-level inputs, DiffSES is able to leverage the simplicity and scalability advantages of symbolic expressions, while also incorporating the strengths of neural networks for feature learning and optimization. Our experiments demonstrate that DiffSES is able to generate symbolic policies that are simpler and more and scalable than state-of-the-art symbolic RL methods, with a reduced amount of symbolic prior knowledge.
translated by 谷歌翻译